Current spatialized audio systems, such as those for home theaters and video games, utilize the “5.1” and “7.1” formats. A 5.1 spatialized audio system includes left and right front channels, left and right rear channels, a center channel and a subwoofer. A 7.1 spatialized audio system includes the channels of the 5.1 audio system and left and right channels aligned with the intended listener. Each of the above-mentioned channels corresponds to a separate speaker. Cinema audio systems and cinema grade home theater systems include DOLBY ATMOS, which adds channels configured to be delivered from above the intended listener, thereby immersing the listener in the sound field and surrounding the listener with sound.
Despite improvements in spatialized audio systems, current spatialized audio systems are not capable of taking into account the location and orientation of a listener, not to mention the respective locations and orientations of a plurality of listeners. Therefore, current spatialized audio systems generate sound fields with the assumption that all listeners are positioned adjacent the center of the sound field and oriented facing the center channel of the system, and have listener position and orientation requirements for optimal performance. Accordingly, in a classic one-to-many system, spatialized audio may be delivered to a listener such that the sound appears to be backwards, if that listener happens to be facing opposite of the expected orientation. Such misaligned sound can lead to sensory and cognitive dissonance, and degrade the spatialized audio experience, and any “virtual reality” or “augmented reality” experience presented therewith. In serious cases, sensory and cognitive dissonance can cause physiological side-effects, such as headaches, nausea, discomfort, etc., that may lead users to avoid spatialized audio experiences, “virtual reality” experiences or “augmented reality” experiences.
Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” (“VR”), “augmented reality” (“AR”), and “mixed reality” (“MR”) experiences, wherein digitally reproduced are integrated into the real world environment of the user and presented as though they are real objects existing in the inertial reference frame of the real world environment. A virtual reality, or “VR”, scenario can involve presentation of digital or virtual image information while occluding the user's view of the real world. An augmented reality, or “AR”, scenario typically involves presentation of digital or virtual image information as an augmentation to the visible actual world around the user (i.e., transparency to other actual real-world visual input). A mixed reality, or “MR”, system also introduces simulated objects into a real-world environment, but these objects typically feature a greater degree of interactivity than in AR systems. The simulated elements can often times be interactive in real time. Accordingly, AR and MR scenarios involve presentation of digital or virtual image information superimposed on the real world environment of the user which is simultaneously visible to the user.
Various optical systems generate images at multiple depths for displaying VR/AR/MR scenarios. Some such optical systems are described in U.S. Utility patent application Ser. No. 14/738,877 and U.S. Utility patent application Ser. No. 14/555,585 filed on Nov. 27, 2014 , the contents of which have been previously incorporated-by-reference herein.
Current spatialized audio systems can cooperate with 3-D optical systems, such as those in 3-D cinema, 3-D video games and VR/AR/MR systems, to render, both optically and sonically, virtual objects. Objects are “virtual” in that they are not real physical objects located in respective positions in three-dimensional space. Instead, virtual objects only exist in the brains (e.g., the optical and/or auditory centers) of viewers and/or listeners when stimulated by light beams and/or soundwaves respectively directed to the eyes and/or ears of audience members. Unfortunately, the listener position and orientation requirements of current spatialized audio systems limit their ability to create the audio portions of virtual objects in a realistic manner for out-of-position listeners.
Current head-worn audio systems (i.e., headphones or earbuds) can be configured to produce spatialized audio. However, these head-worn audio systems are disposed either on or in the listener's ears. As such, current head-worn audio systems transmit tactile signals to the listener's brain resulting from the physical contact between the head-worn audio systems and the listener's ears. These tactile signals can lead to a psychoacoustic effect that suggests to the listener that the sounds generated by the head-worn audio systems emanate from a short distance to the listener's ears. Consequently, spatialized audio produced by current head-worn audio systems may appear to emanate from a location different from that of the virtual object. Further, current head-worn audio systems do not address the user position and orientation requirements of current spatialized audio systems.
With improvements in home theater systems, traditional cinemas are losing audiences to home theaters. Accordingly, filmmakers and film companies are searching for improvements in motion picture technology. In a similar technology space, mixed media systems such as those found in theme park rides (i.e., DISNEY'S STAR TOURS) can add real life special effects such as lights and motion to 3-D film and spatialized audio. However, such systems are prohibitively expensive and are not individualized. Moreover, such mixed media systems do not address the inherent user position and orientation requirements of current spatialized audio systems.
Users of 3-D mixed media systems are typically required to wear glasses that facilitate perception of 3-D imagery. Such glasses may contain left and right lenses with different polarizations or color filters, as in traditional anaglyph stereoscopic 3-D systems. The 3-D mixed media system projects overlapping images with different polarizations or colors such that users wearing stereoscopic glasses will see slightly different images in their left and right eyes. The differences in these images are exploited to generate 3-D optical images.
Similarly, spatial audio systems can be worn on the heads of users. However, the above-described psychoacoustic effect reduces the effectiveness of current head-worn spatial audio systems, by affecting the perceived position of virtual audio sources rendered by the systems.
In summary, current spatialized audio systems have listener position and orientation restrictions, and psychoacoustic effects that limit their effectiveness in rendering spatial audio corresponding to a virtual object for moving listeners and for pluralities of listeners in a variety of positions and orientations. In addition, traditional 2-D and 3-D films, 2-D and 3-D video games and mixed media systems can benefit from individualization including spatialized audio for pluralities of moving users/viewers/listeners that address user position and orientation restrictions, and psychoacoustic effects.